Online Learning in Stochastic Games and Markov Decision Processes

In their standard formulations, stochastic games and Markov decision processes assume a rational opponent or a stationary environment. Online learning algorithms can adapt to arbitrary opponents and non-stationary environments, but do not incorporate the dynamic structure of stochastic games or Markov decision processes. We survey recent approaches that apply online learning to dynamic environments where the assumptions of rationality and stationarity are invalid. The emphasis is on the connections and differences between different approaches arising from different modeling assumptions and performance baselines.